With increased economic globalisation and the desire to increase productivity, coupled with improved network communications and the impact of the Internet in particular, the world has become increasingly dependent on the ability to retrieve data that is required at a data processing apparatus from elsewhere in a global network. The required data may include data files such as sound or video, executable files, BLOBs (binary large objects) from databases, e-mail attachments, etc. For example, e-mail communications and access to Web pages are essential for daily business in a vast range of industries, and software patches and upgrades are made available for download via the Internet to avoid the cost and delays of distributing diskettes or CD-ROMs.
However, with increased use of home computers, mobile communications and mobile data processing devices, much of this network traffic is exchanged across relatively low bandwidth communications channels. Additionally, many organisations connect their local area networks to the Internet via proxy servers for reasons of cost, security and management efficiency. When the proxy server is heavily used, the capacity of the proxy server or its communication channels may limit communication throughput even if a relatively high bandwidth channel is available. Furthermore, a high bandwidth connection between a local computer and its neighbours within a network does not imply that all of the required links between the start and end points of a network communication can match that bandwidth—bottlenecks and consequent delays can arise anywhere in the network.
With more and more applications being made available for access from anywhere in the world, Internet communication traffic has become excessive. Typical application response times can increase as a result, from milliseconds or seconds to seconds or minutes. This reduces the productivity of computer users and reduces the useability of the applications. The ‘applications’ in this context may include, for example, services provided by Web servers, application servers, mail servers, ‘groupware’ applications, ‘instant’ messengers that allow files to be exchanged, automated software installers or databases.
A great deal of the data flowing across congested Internet connections is repetitious. It is common for several people within the same department of an organisation to download the same data via the same proxy server. Furthermore, individual users often download a second copy of data that they retrieved previously—such as when a small part of the data has changed or when a program installation process was only partially successful. In some cases, a user repeats retrieval of data because the user cannot recall where data was saved. Although automated caching of data is known, the data held in a cache is typically only available to the specific application that cached the data. Furthermore, although a Web browser may have cached material from a Web site, if a different URL is used to access the same material the Web browser will fetch a new copy of the material.